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This paper examines two factors that have been shown in previous literature to enhance students’ 
interest in learning mathematics — personalization of problems to students’ interest areas, and the 
addition of visual representations such as decorative illustrations. In two studies taking place within 
an online curriculum for middle school mathematics, students receive problem types that vary with 
respect to these factors. Results show that while these interest-enhancing interventions may benefit 
students in the short-term as they solve modified problems, there is little evidence they allow students 
to understand key mathematics concepts more deeply. 
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Research has revealed how many students tend to disengage with mathematics over adolescence 
(Fredicks & Eccles, 2002; Frenzel, Gotez, Pekrun, & Watt, 2010), and increasingly have difficulty 
seeing the relevance of mathematics to their lives (McCoy, 2005). Accordingly, research on how 
interest can be activated and maintained in classrooms has become prevalent in educational 
psychology. Some interventions to enhance students’ interest in curricular materials include adding 
colorful illustrations (Durik & Harackiewicz, 2007), personalizing instruction to students’ out-of- 
school interests in topics like sports or music (Walkington, 2013), and giving learners choice in their 
learning activities (Patall, 2013; Potvin & Hasni, 2014). Visual representations that enhance interest 
may promote persistence and focus of attention, and sometimes they directly provide mathematical 
information to support students. Personalizing problems may also enhance interest, and allow 
learners to draw upon prior knowledge of concrete, relatable situations. While these interventions 
have shown promise for eliciting interest, consideration is not always given to the cognitive 
implications of the modifications. Specifically, features designed to enhance interest may distract 
learners from grappling with the mathematical concepts that should be the central focus, a 
phenomenon known as the seductive details effect (Harp & Mayer, 1998; Lehman, Schraw, 
McCrudden, & Hartley, 2007). Also, if learners become accustomed to these kind of interest- 
enhancing supports, they may struggle in situations where they must solve abstract mathematics 
problems. Research on desirable difficulties (Schmidt & Bjork, 1992) suggests that learners may 
benefit more in the long term from a lack of support in their learning environment, as this forces 
them to grapple with concepts and make important connections on their own. Further, research on 
simple symbols suggests that teaching concepts using abstract formalisms — rather than concrete 
applications — allows for better transfer of learning of the underlying mathematical ideas (Sloutsky, 
Kaminski, & Heckler, 2005). 

Here we examine interest-enhancing interventions — personalization, choice, and the use of 
illustrations — in a curriculum for 6" grade math. We examine the short term effects of these 
modifications — whether the interest enhancement is supportive or seductive — and well as the long 
term effects on student learning — whether the interest enhancement is a crutch or a scaffold. 


Literature Review 


Visual Representations 
Considerable research shows that learners benefit from visual representations (e.g., Mayer, 
2009), and effectively using visual representations improves problem solving (Woodward et al., 
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2012). Research into the effects of diagrams, one form of visual representations, in middle school 
mathematics includes topics such as the creation of schematic diagrams for spatially-oriented 
arithmetic word problems (Boonen et al., 2014), the use of diagrams in algebra word problems 
(Booth & Koedinger, 2012), and the use of diagrams in proportional reasoning tasks (Jitendra & Star, 
2012). While there are some positive findings for diagrams, overall findings are mixed (Booth & 
Koedinger, 2012). More research is warranted to evaluate when, where, and how such diagrams will 
be most helpful. There are also mixed findings regarding the impact of decorative illustrations 
(Berends & van Lieshout, 2009; Jaeger & Wiley, 2014). Considerable support (Harp & Mayer, 1998; 
for a review, see Rey, 2012) has been found for the coherence principle (Mayer, 2009), which 
suggests that removing interesting but irrelevant information contained in purely decorative 
illustrations fosters learning. There are also mixed findings for decorative illustrations that contain no 
mathematical information, but that illustrate the context of a problem — they have been found to have 
no influence on problem solving (Dewolf, van Dooren, EvCimen, & Verschaffel, 2014) or be helpful 
(Elia & Philippou, 2004). Here, we use illustrations that have diagrammatic features that give 
mathematical information, as well as purely decorative illustrations that illustrate or do not illustrate 
the story context. 


Personalization and Choice 

We define personalization as an instructional approach that connects math tasks to students’ out- 
of-school interests in broad topics such as sports, shopping, and video games (Walkington, 2013). 
Research on personalization in mathematics has yielded mixed findings — there is some evidence that 
personalization carefully accomplished through student interest interviews and open-ended surveys 
can promote achievement gains in algebra (Walkington, 2013), and that deeper and more authentic 
personalization can be more effective than shallow approaches in which a few words are simply 
swapped out of story problems and replaced with words related to an interest topic (Walkington & 
Bernacki, under review). Other research suggests that even very shallow attempts at personalization 
can promote student performance and learning (Anand & Ross, 1987; Cordova & Lepper, 1996). 
However, more recent studies have challenged whether personalization is even worth doing if there is 
not deep engagement with the actual quantitative knowledge that students actually use as they pursue 
their interests (e.g., Fancsali & Ritter, 2014). Finally, complementary research also shows that the 
facilitation of learner control or choice in a learning environment has the potential to enhance interest 
and motivation (Linnenbrink-Garcia et al., 2013; Patall, 2013; Potvin & Hasni, 2014), as well as 
learning (Cordova & Lepper, 1996). 


Research Purpose 

In prior work (Walkington, Cooper, & Howell, 2013; Walkington, Cooper, Nathan, & Alibali, 
2015), we found that personalization and illustrations used in worksheets covering middle school 
mathematics concepts enhanced students’ interest, but did not affect their performance. Here, we 
expand this research by examining several different types of illustrations and several different 
approaches to personalization in an online, adaptive mathematics curriculum, and examine the effect 
not only on short-term performance but on long-term learning. Our research questions are: How do 
different types of illustrations impact students’ accuracy on the illustrated problems and their 
performance on a post-test without illustrations? How do different approaches to personalization 
impact students’ accuracy on personalized problems and their performance on a non-personalized 
post-test? 


Method 
Both studies took place within the Reasoning Mind 6" grade curriculum. Reasoning Mind is a 
mathematics blended learning system developed by a nonprofit organization. Within this system, 
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students use computers during their math class while their teacher conducts targeted interventions 
with students who are struggling. The student is immersed in a lesson environment that includes a 
tutor character, two other student characters, and a virtual blackboard. The student characters make 
common mistakes which the real student is asked to correct, they help the real student when he or she 
gets stuck, and they interact with the real student and the tutor in ways that are intended to promote 
beneficial mathematical attitudes and beliefs. Both studies took place in two small urban middle 
schools in Texas. Study 1 involved 265 6" grade students, while Study 2 involved 223 6" grade 
students (demographics in Table 1). 


Table 1: School Demographics and Performance 


Demographic Group % in School A % in School B 
Hispanic/Latino 33.1 97.4 
Asian 6.0 0 
Black or African American 17.3 is 
Native Hawaiian or other Pacific Islander 0.4 0 
White 36.7 1.1 
Two or more races 6.0 0 
Economically disadvantaged 69.8 92.2 
Limited English Proficient (LEP) 10.9 PAS Fe. 
2014 STAAR Mathematics Passing Rate 30,2 24.3 


Study 1: Visuals 

Study 1 included 4 fractions problems in Lessons 88 and 89 of the curriculum. All problems gave 
a fractional measurement (e.g., 3/8 of a meter), and then described what part of a whole this 
measurement was (e.g., was 21/40 of the whole length). The students then had to solve for the whole. 
Students were randomly assigned to one of five conditions (see Figure 1): 1) a control condition with 
no illustrations for the 4 problems, 2) a condition with diagrammatic illustrations that contained 
mathematical information in the form of a number line (e.g., in Figure 1, the cord could be thought of 
as a number line with a certain amount of the whole line indicated by being ““chewed”’) or a shaded 
area model (e.g., an illustration showing how much grass in a whole field had been mowed), 3) a 
condition with contextual illustrations that simply showed part of the story context and contained no 
mathematical information, 4) a condition with misleading diagrammatic illustrations that contained 
incorrect mathematical information (e.g., in Figure 2 the illustration makes it look like most of the 
cord is chewed, when the answer shows that only a small portion has been chewed), and 5) a 
condition with irrelevant illustrations that had nothing to do with the story context. The misleading 
illustration condition was added based on the observation that some of the illustrations already in the 
curriculum were actually misleading — for example, there would be a problem about a snake who had 
one third of his length wrapped around a pole, but the illustration would display a snake completely 
wrapped around a pole. The diagrams were designed to be supportive rather than essential (1.e., the 
problem could be solved without looking at them). This is common in math curricula, and it allowed 
for the problems to still be solvable in conditions where the visuals were purely decorative. 


Study 2: Personalization & Choice 

Study 2 involved two more extended problem scenarios involving rates in Lesson 103 of the 
Reasoning Mind curriculum. Students were randomly assigned to one of four conditions: 1) a control 
condition with the standard story problems already in the unit (shown in Figure 2), 2) a condition 
where the problem topic is modified and assigned based on students’ highest reported interest across 
four personalized topics (sports, food, shopping, and video games) on an interest survey given to all 
conditions, 3) a condition where students are randomly assigned to one of the four personalized 


Wood, M. B., Turner, E. E., Civil, M., & Eli, J. A. (Eds.). (2016). Proceedings of the 38th annual meeting of the 
North American Chapter of the International Group for the Psychology of Mathematics Education. Tucson, AZ: 
The University of Arizona. 


Articles published in the Proceedings are copyrighted by the authors. 


Curriculum and Related Factors 92 
versions of the problem, and 4) a condition where the student is able to choose the problem topic 
from the four personalized topics before working on the problems. 


Diagrammatic Illustration Contextual Illustration 
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Misleading Illustration Irrelevant Illustration 
Figure 1. Example of conditions in Study 1| (there was also a “No Illustration” condition). 


Depending on condition, students were given one of five versions of the same problem. In each 
version, the numbers and question remained the same, but the topic was changed. The control 
problems are in Figure 2. The other four versions were personalized and changed the topic to sports, 
shopping, video games, or food. For the first problem (Figure 2, top), the personalization was 
shallow and involved swapping out “books” for another noun — footballs, lollipops, necklaces, and 
crystals. For the second problem (Figure 2, bottom), the personalization was deeper as more words 
were swapped — each of the locations was replaced with a setting that someone who engaged in the 
personalized topic would be interested in. For example, the video game variation discussed Kayla 
traveling to an Enchanted Forest, Dragon Cave, and Wizard’s Tower while playing a video game. 
Readability factors were kept consistent among personalized variations and between personalized 
and control group problems, as was the presence and type of illustrations. For the first problem, all 
versions had two illustrations — one with an image of the object that had been swapped out for the 
pile of books (e.g., footballs), and one with a pile of dollar bills. For the second problem, all versions 
had the same diagram, but the images of each location were redrawn to match the locations given in 
the modified story (e.g., a dragon cave). 
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Let’s use a table to investigate how the number of books I 

buy relates to the cost. I was at the bookstore the other day. 

I bought 1 book, which cost 4 dollars. Then I decided that 

I wanted two copies of the book. How much will two = 
books cost? ty 


Kayla walks at a speed of 50 meters per minute. It takes Kayla 

10 minutes to walk the distance of 500 meters to school. After a 

school, it takes her 20 minutes to walk the distance of 1000 

meters to the library. After the library, Kayla walks to volleyball 

practice. The distance to volleyball practice is three times the 

distance she walks to school. If Kayla’s speed did not change, - 
and the distance increased by a factor of 3, then what do you =—- 4» i | CO 
think happened to the time? 


Figure 2. Control (non-personalized) problems utilized in Study 2. 


Data Analysis 

In both studies, data were analyzed using linear regression models that predicted accuracy on 
each of the intervention problems and percentage of problems correct on the post test. The accuracy 
for individual problems was computed by assigning one point to each prompt the tutor asked students 
to type a numerical response into, and averaging performance across all such prompts. Predictors 
included Condition (4 or 5 levels), as well as controls for prior knowledge differences between 
conditions. We had slightly different prior knowledge data available in each study — in Study 1, we 
included performance on the previous unit test and previous lesson as controls. In Study 2, rather 
than prior test performance, we had available Guided Study Accuracy (an overall in-tutor measure of 
knowledge). The post-test measures for each study also varied slightly. In Study 1, there was a quiz 
immediately after the two intervention units that could be utilized. However, in Study 2 there was 
just a unit test available in the software that covered all the proportion lessons in the tutor, and no 
subsequent quiz. Although we used the unit test for long-term learning outcomes in Study 2, we also 
did supplementary analyses of individual items on the unit test that better aligned to the units where 
the intervention was placed. However, when analyses were done using individual items or sets of 
individual items, no additional significant effects were detected — thus we use the score on the entire 
test. 


Results 


Study 1: Visual Representations 

Results showed that for the second problem, which is the one used as an example in Figure 1, 
having diagrammatic illustration significantly improved performance, compared to the other four 
conditions (B = 14.97, p = 0.003). However, there were no differences between the different 
conditions for models predicting performance on the other problems (ps > 0.1). The differing 
behavior of problem 2 was also shown by a problem by condition interaction when the data was 
analyzed with all the problems together. The illustrative diagrams were significantly more beneficial 
to performance on problem 2, compared to the other 3 problems (Bs = 12.82, 13.95, 12.54; ps = 
0.014, 0.007, 0.014). Anecdotally, the diagrammatic illustration for Problem 2 may have been 
particularly effective because Pawthagoras is a fun and well-liked character within Reasoning Mind 
so students may be more likely to focus on the information in the diagram. 

Also examined was performance on the post-quiz at the end of Lesson 89. Results showed that 
students who had received no visual images for the four problems while receiving their learning 
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materials significantly outperformed the other conditions on the quiz (B = 12.03, p = 0.030). Thus 
while there was some evidence from one of the problems that diagrammatic illustrations allowed 
students to perform better in the short term, in the long term, not having any visual representations at 
all facilitated post-test performance. 

Data were also available for students’ response to the prompt “How much did you like this 
lesson?” on a 5 point scale (Terrible, Bad, OK, Good, Great) for lessons 88 and 89. Students’ two 
ratings were averaged, and entered into a linear regression model predicting average lesson rating 
based on Condition as well as their ratings of the four prior lessons as controls. Data for 49 students 
were missing due to the student not answering the prompt for either of the lessons. Results showed 
that students who received illustrations that contained no mathematical information (1.e., Contextual 
Illustrations and Irrelevant Illustrations) rated that they liked the lessons significantly more than other 
students (B = 0.328, p = .0185). Thus illustrations added purely for decorative purposes seemed to 
enhance students’ ratings of the lessons, although the effect was relatively modest (0.3 on a 5-point 
scale). 


Study 2: Personalization & Choice 

Results for performance on the different versions of the shallow personalized problem (Q1 in 
Figure 2) showed no significant differences by Condition (ps > 0.1). Results for the different versions 
of the deep personalized problem (Q2 in Figure 2) showed that students who received personalization 
by choosing immediately before the problem significantly outperformed the control condition on this 
problem (B = 11.82, p = 0.005), and students who received personalization based on a survey also 
outperformed the control (B = 8.76, p = 0.035). The benefit of receiving a personalized problem 
randomly assigned over the control condition did not reach significance (p = 0.088); no further 
contrasts were significant (ps > 0.1). Thus personalization, whether accomplished through choice or 
survey assignment, boosted students’ performance relative to solving a non-personalized problem. 

On the unit test, although students who received personalization and choice numerically scored 
highest, this difference did not near significance when compared to the control condition (p = 0.305). 
The only significant contrast for this model suggested that students who receive personalization and 
choice score significantly higher on the post-test than students who receive a random personalized 
problem (B = 12.92, p = 0.033). Thus there is further evidence that personalized versions of problems 
in the absence of some sort of intelligent selection system to assign them based on student interests 
are not particularly useful. However, there is little evidence to suggest that personalization acted as a 
crutch that hindered students’ performance when later solving non-personalized problems. 


Discussion & Conclusion 

Study | gave fascinating results as to how visual representations interact with students’ 
performance, attitudes, and long-term learning. Diagrams that contain mathematical information 
enhanced student performance for one of the problems, but only in the short term. This suggests that 
for some students, these representations may be a crutch when taking a post-assessment with no 
visuals. Purely decorative visuals were liked better by students, but there was no evidence they 
enhanced performance or learning. Finally, the absence of visuals altogether seemed to allow 
students to learn best from the materials. However, while learning is certainly important, having 
students enjoy working in their math curriculum, rather than find it tedious or boring, is an outcome 
that should not be dismissed as irrelevant. Study 2 suggests that personalization accomplished 
through intelligent selection of interest-based problems (either through a survey or learner choice) 
can enhance performance in the short term, but only for problems where the personalization is more 
deeply and thoughtfully accomplished. In the long term, receiving personalization did not show any 
advantage over receiving non-personalized versions of problems. Attitudinal measures were not 
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available in Study 2, so it would be interesting to observe in future work how different approaches to 
personalization impact lesson ratings. 

Taken together, the results from both studies suggests that curriculum designers need to think 
critically about the outcomes they most value, and how those outcomes may be at odds with each 
other when considering interest-enhancing interventions. In some cases, interest enhancements that 
boost performance and interest in the short term as students are solving problems may not transfer to 
assessments of learning, and could even potentially harm learning compared to materials without the 
enhancements. Our results for visuals suggest that the absence of visuals may be a desirable 
difficulty that forces students to become accustomed to solving unadorned problems without visual 
supports or cues. Our results for personalization suggest that the addition of interest-based content is 
not seductive or distracting, and that it may help in the short term if it is well-matched to learners’ 
actual preferences. However, the type of personalization we implemented here where words were 
simply swapped out of the stories may not be effective for promoting long-term learning. Instead, if 
long-term learning is the goal for personalization, research suggests that considering how students 
actually use quantities in their day-to-day life when pursuing their interests might be most effective 
(Walkington & Bernacki, 2015). Both studies also suggest that all approaches to interest- 
enhancement are not created equal — for visuals, whether the visual contains mathematical 
information is key; for personalization, the depth of the personalization and how problems are 
assigned to students are important. This suggests that when compiling data from multiple studies on 
interest enhancements, it is critical to pay close attention to how the enhancement was actually 
implemented in the curriculum. 

Students’ interest in learning mathematics can wane over the middle grades, and curriculum 
developers are increasingly drawn towards quick solutions to attempt increase student engagement 
with their materials. Many of these solutions can involve considerable cost to the curriculum 
developer (e.g., hiring an artist or writing multiple versions of each problem), thus it is important to 
consider how motivational enhancements impact students’ understanding of mathematical ideas. 
Future research should delineate the most effective interest-enhancing supports for different profiles 
of learners, and for different mathematical content areas. 
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